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ABSTRACT 

Two hundred and thirty-two articles, research 
rp>ports, graduate dissertations, and books, published between 1953 
and 1973, are listed in this bibliography for scholars, educators, 
and researchers. The bibliography is a supplement to a report 
concerning the association between educational outcomes and 
background variables. A wide range of studies is listed, including 
investigations of personal, attitudinal, and environmental factors 
which affect educational outcomes, and several statistical studies 
educational outcomes and background variables are highlighted. The 
relevance of thise studies to the objectives of the National 
Assessment of Educational Progress is emphasized. (Author/SE) 
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Pref aoe to the Appendix 



This report presents a bibliography of papers examined 



by the project staff as part of a review of selected literature 
on association between educational outcomes and background vari- 
ables. The user of this appendix voluiao should refer to the 
main report, which is to be published as a monograph by the Na- 
tional Assessment of Educational Progress, for a description of 
the project which gave rise to this bibliography. It is neither 
comprehensive nor discriminating, but may serve as a useful 
starting-point for other researchers. Following the bibliography 
is a set of abstracts of documents having particular relevance 
to the subject matter of the main report. It is assumed that 
readers of the main report will want to refer to thes^e abstracts 
to judge the statistical competency of some of the results cited. 



Edward C, Bryant 
Project Director 
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2(P 



David J. Armor 



"School and Family Effects on Black and White Achievement: 
A Reexamination of the USOE Data" 

In Mosteller, Frederick and Daniel P. Moynihan, eds., 

On Equality of Educational Opportunity , New York: 
Vantage Books, 1S72. 

Objectives : 

1, To examine, by means of an independent assessment- 
analysis of the complete data set on elementary 
schools covered in the Coleman-USOE study, the 
major conclusions of the "Coleman study." 

2, The conclusions are stated as: a) blacks appear to 
have just as adequate school facilities as whites in 
most parts of the country; b) aside from the dis- 
tribution of school quality, the effects of school 
staff cind facilities on achievement do not seem large 
for either blacks or whites; c) school factors 

are not as important as the effects oi family and 
community factors. 

3, To examine the Colemar. data against four methodo- 
logical/conceptual problems that bear upon policy 
conclusions drawn from the USOE data: 

Methodological: 1) inherent validity/reliability 
limitations of self-administered principal/teacher/ 



student questionnaires; 2) the high non-response 
rate of these self -administered questionnaires. 

Conceptual: 1) definitions of school output, in 
terms of effects schools are supposed to have on 
the students; 2) the limitation of cross-sectional 
studies with respect to .causal relationships. 

4 . To develop a conceptual model of the educational 

process, for the purpose of evaluating the equality 
of educational opporunity. 

Study Design: 

1. Study was a re-analysis of data collected by the 
Coleman study on elementary schools nationwide. 
Elementary schools were selected because 1) the 
response rate for elementary schools was good in 
the Coleman study; 2) the student bodies of elementary 
schools are relatively homogeneous with respect to 
measures of community input factors; 3) the three 
grades — 1st, 3rd, 6tn — assessed by Coleman permits 
a comparison of student achievement over a six-year 
period; 4) the assumption was made that student 
experiences during the elementary school years can 
have more important effects on dchievement than 
later schooling. 
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2. The Coleman sample of elementary schools was further 
refined, eliminating those schools with grades 1-12 
and those elementary schools with student bodies 
with 25% or more pupils other than black or white 

in race. Sample was of 1623 elementary schools, 
1623 providing data on sixth graders and 880 pro- 
viding data on first graders, 

3. Indices were constructed for a model of educational 
process: School Input measures; School Output 
irsasures; and Commuiiity Input measures, 

4. Data presented for aggregate school level, schools 
classified by race: black schools were schools 
with more than 50% blacks in the .student body; 
white schools were schools with more than 50% 
whites in the student body, 

5. Data presented by geographic region and size of 
place: Metropolitan (=SMSA) and Nonmetropolitan , 



6. Sampling Procedures: a) Data were from the Coleman- 
USOE Report; the total sample of elementary schools , 
surveyed by Coleman was further refined by elimi- 
nating schools with grades 1-12 and schools with 
student bodies with 25% or more pupils other than 
black or white in race, b) Weights: A weighting 
factor was computed for each school, which varied 
inversely with the probability with which that 
school was sampled; a weighting factor was computed 
for each grade, which reflected the grade enrollment 
and the proportion of among-school variance for 
various school factors. 
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VI. Instruments and Measures: 

1. Indices were computed from data supplied by the USOE 
report : 

-Principal questionnaire 
-Teacher questionnaire 
-Teacher verbal achievement test 
-Student questionnaire 

-Student verbal achievement tests, 1st and 6th grades 

2, Indices were reported on an aggregate school level: 
a. School Input indices: 

1. school facility index, computed from list 
of school facilities reported available 
by principal 

2. teacher professional background, averaging 
the indices of all teachers in the school; 
derived from teacher questionnaire 

3. teacher general verbal achievement, aver- 
aging the scores of all teachers in the 
school; derived from the teacher verbal 
achievement test 

4. annual per-studcnt expenditures, as mea- 
sured by averaged teacher salaries divided 
by total number of students in the school. 
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b. Student Output indices: 

1. verbal achievement scores of sixth grade 

students, averaging the scores of students 

in the majority race only 
2 • verbal achievement scores of first grade 

students, averaging the scores of students 

in the majority race only 

c. Community Input indices: 

1. structure of students' families, computing 
the percent of sixth grade students of 
both parents living at home 

2. occupational status of the father, com- 
puting the percent of sixth grade students 
of both races whose fathers are white 
collar workers 

3. family life style, averaging the indices 
of all sixth grade students of both races, 
in terms of appliances in the home 

4. education of the students' parents, 
averaging the indices of all sixth graders 
of both races, in terms of parents' educa- 
tional backgrounds. 

The basic unit in the data presentations was the 
school, and all data was aggregated on the school 
level, 

-School Input indices were computed for over-all 
school characteristics, with the schools classified 
as b lack (more than 5 0% black student body and as 
white (more than 50% white student body) . 



-School Output indices were aggregated only for 
black students in black schools and only for white 
students in white schools 

-Conununity Input indices were aggregated over both 
races in the school, for sixth grade students only. 



Variables Studied: 



1. Intermediate variables: 

1. School facility 

2. Teachers' professional background 

3. Teachers' general verbal achievement 

4. School per capita student expenditures 

5. Structure of family 

6. Occupational status of father 

7. Family life style 

8. Education of the students' parents 

2. Outcome variables: 

1, Sixth grade students' verbal achievement 



Statistical Procedures: 

1. Correlation analyses 

2, Multiple regression analyses 
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Relevance to the NAEP Study 



There has been substantial criticism of the analysis of 
the Coleman data. This paper rexamines the analysis, 
using the school as the unit of analysis. The school 
variables analyzed are particularly relevant to this 
study. 
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Jerald G. Bachman et at 



'fouth in Transition, Vol, 1: Blueprint for a Longitudinal 
Study of AdolsQQcnt Boys; Vol. 2: The Impact of Family 
Baakground and Intelligence on Tenth-Grade Boys, 

III. Survey Research Center, Institute for Social Research, 
University of Michigan, Ann Arbor, Michigan, 1967 (Vol. 1) 
and 1970 (Vol. 2) . 

IV. Objectives: 

1. To assess a number of changes that occur during 
adolescence, including dimensions of self-concept, 
values, attitudes, plans, aspirations and behaviors 
and to determine how these relate to environment 
characteristics and personal characteristics, 

2. To measure the background and personality charac- 
teristics of a sample of adolescent boys at three 
different times during the three year period begin- 
ning at tenth grade entrance and ending one year 
following high school graduation. 

3. To determine the characteristics of th<i school and 
work environments of these boys in order to assess 
person-environment fit. 

4. To study the school as formal organization in 
order to identify the orrranizational characteristics 
that relate to the drop-nut phenomena. 
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Design of Study s 



1. Longitudinal 

2. Combines depth of longitudinal study with the 
breadth of a nation-wide sample, 

3. Covers a three year period with interval data 
collection as follows; October, 1966, March, 1968 
and November, 1969. 

Description of Sample: Sample I: Probability sample 
!• Size of sample: 

a. 2200 boys 

b. 87 schools 

2* Population: 

a. All boys in the 10th orade in public high 
schools in the United States, 

b. All public high schocls in the United States 
as of Sumner, 1964 , w^.ich had at least 15 boy 
in the 10th grade, T)is excludes less than 
2,5% of all 10th qradc public school boys. 

3, Sex: males only 

4, Racial composition: 1912 whites; 256 blacks; 45 
other 

a. The sample of blacks '-'ere sub-rlividGd into 
groups based on Ircat. on anrl .-chool . 



1. 

2. 
3. 



73 attended integrated schools 

72 attended northern segregated schools 

111 attended sourther segregated schools 



b. Number of schools attended by blacks 

1, 183 students were in 9 schools (segregated) 

2, 73 students were in 25 schools (integrated) 

Religious composition: 63 percent Protestant; 20 
percent Roman Catholic and Eastern Orthodox? 3 
percent Jewish; and 14 percent Other and missing 
data. 

Sampling procedure: 

a. Three step process involving stratification and 
clustering included: 

1, Separation of the United States into 88 
strata. The Survey Research Center has 
developed a sampling framework which 
divides the United States, excluding 
Hawaii and Alaska, into 88 strata with 
each stratum representing about 2 million 
people. 62 of these correspond to separate 
counti( ; the rest are grouped into 12 
major metropolitan areas. 

2. A random selection of a single school in 
each of the 8 8 strata was obtained. The 
probability of the selection of any school 
was proportional to the estimated number 
of 10th grade "lales. 



3, A random sample of thirty boys was ob- 
tained within each school. 

b. Of the 88 schools drawn according to design, 
71 responder! affirmatively (81 percent) . 

c. Replacement schools from the sample areas where 
schools had responded negatively were secured. 
16 out of 17 accepted. 

d. Final response rate was 97 percent. 



Description of Sample: Sample II: Supplementary, 

discretionary sample of out- 
standing schools 

1. Size of sample: 

a. 300 boys 

b. 10 schools 

2. Population: 17 schools selected for excellence in 
one or more of the following areas: academic 
curriculum, organizational innovation, student- 
faculty relations, vocational preparation or 
promoting student mental health. 

3» Sampling procedure: Discretionary. Sample popula- 
tion was selected by a panel of experts in education. 
The names and qualifications of the experts who 
did the selecting and those who were selected is 
not given. The final sample consisted of those 
schools which accepted the invitations. 
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4. Rationale for supplementary sample: the representa- 
tive random sample may not include many outstanding 
schools. It was felt that in a study designed to 
show what school environments can do, as well as 
what they typically do, such a defect might be 
serious, and to insure that there would be a suffi- 
cient number of outstanding schools, a s^^^^cial 
supplementary sample should be chosen. 

VIII. Instruments and Measurement (Time 1): 

1. A student questionnaire developed by the staff 
(using some existing instruments and devising 
additional items) designed to measure: 

a. Affective states 

(e.g., self-esteem, depression, resentment, 
guilt, impulse to expression, life satisfaction) 

b. Personality dimensions 

(e.g., self-development and self-utilization, 
need for social approval, fear of failure, test 
anxiety, flexibility) 

c. School opinions 

(e.g., school influence description, attitudes 
toward teachers, attitudes toward school, proba- 
bility of dropping out of school, deviant be- 
havior in school) 

d. Values and attitudes 

(e.g., cultural values, job attitudes, internal 
vs. external control, political attitudes and 
information) 
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e. 


Life outside of school 




(e.g., social and dating behavior, family re- 




lationships, physical health and appearance, 




political and religious? preference, socio- 




economic status, participation in activities) 


f. 


Delinquent behaviors 


A structured interview designed by the project staff 


using existing instruments and their own questions 


were 


conducted to gather data on the following: 


a. 


Peer relationships 


b. 


Self-concept of school ability 


c. 


General happiness 


d. 


Motives 


e. 


Job history and financial status 


f. 


Future plans, interpersonal influence 


g. 


Person-environment fit, self-identity dimensions 


h. 


Dropping out and reasons for doing so 


t 

1. 


Paragraph comprehension test 


■ 


IQ: from Quick Test of Intelligence developed 




by Ammons and Ammons (1962) 



3. A Group Test Battery used to measure academic 

abilities and aptitudes consisted of the following: 

a. Matrices, patterned after Raven's Progressive 
Matrices. It is thought to be relatively free 
from cultural and educational bias (Raven, 1951) . 
The test is considered a useful predictor for 
individuals who have good reasoning ability but 
who may have difficulty in school achievement 
because of non-intellectual factors. 
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b. Gates Test of Reading Comprehension from the 
Gates Reading Survey, Teachers College, Colianbia 
University used to measure reading achievement. 

c. Anagrams, by Guilford: a verbal task which 
measures divergent thin^-.ing, 

d. Maze tracing, a sub-test in the performance 
section of the Wechsler-Bellevue , was used as 
a measure of intelligence. 

e. General Ability Test Battery (GATB) , Pare J: 
Vocabulary and Part I: Arithmetic Reasoning 
developed by the U.S. Employment Service de- 
signed to measure general intelligence. 

f. Hidden patterns, a test obtained from the Kit 

of JTognitive Factors, developed by French et al, 
use'd as an indicator of cognitive style: of 
field independence-dependence. 

g. The Job Information Test, a set of items de- 
signed by K^ren E. Paige and Jerald G. Bachman, 
was used to measure knowledge about a wide 
variety of occupations. 



Instruments and Measurement (Time 2 and 3) 

1. Will be essentially the same as those used for Time 
1 with thes omissions: 

a. Aptitudes and abilities 

b. Demographic information 

c. Future plans (Time 3 only) and these items 
marked "optional" 



1. School motivation 

2. Flexibility 

3. Self-utilization opportunities (Time 3 only) 

4. Family relationships 

Instruments for measuring school environments: a 
special questionnaire was designed to obtain data 
about each school. Details concerning the data 
collection were not presented except that data would 
be collected from school staff. The topics that 
were to be covered in the questionnaire were as 
follows : 

1. Inputs required by school 

(e.g., personnel, operating funds, buildings 
equipment and maintenance service) 

2. Processes of allocation of inputs 

(e.g., extent to which staff of school is in- 
fluential in procurement and allocation) 

3. Resources currently held by the school 
(e.g., personnel, students, building grounds, 
equipment) 

4. The School's Role System 

(e.g., stability of roles, changing role system, 
changing role occupants, balance between role 
prescriptions and individual role elaboration 
among teachers, evaluation of the system of 
roles, socialization of new members, evaluation 
of role performance, rewards and penalties, 
sources and nature of control of inputs, com- 
position of role sets, maintenance of role 
occupancy) 



5. Openness and closeness 

(e.g., changes in students: do they have in- 
creased skills and knowledge?; have they for- 
mulated career plans?; have they "matured"?) 

6. Additional properties of organizational structure 
(e.g., size, degree of specialization, group 
norms, group structure) 

Inctrumonts for measuring work environments: not 
described. 

Variables Studies: 
Background variables: 



1. 


Student 


general background 


2. 


Student 


aptitudes and abilities 


3. 


Student 


physical characteristics 


4. 


Student 


job history 


5. 


Student 


history of schooling 


Intermediate 


variables : 


1. 


Student 


motives 


2. 


Student 


affective states 


3. 


Student 


self-concept 


4. 


Student 


values 


5. 


Student 


attitudes 


6. 


Student 


plans 


7. 


Student 


behaviors 


8. 


Student 


role characteristics 


9. 


Environmental variables — family 


10. 


Environmental ariables — interpersonal influences 


11. 


Environmental variables — school 


12. 


Environmental variables — community 



Outcome variables: 

1. Levels of vocational preparation 

2. Levels of aspiration 

3. Levels of skill, knowledge 

4. Levels of self-esteem and affective states 

5. Levels of satisfaction, self-utilizations, self- 
development 

6. Levels of self-concept 

7. Levels of attitudes 

8. Levels of Motivation 

9. Levels of behavior 

10. Levels of amount of role conflict 

Statistical Procedures 

1. Three broad strategies carried out sequentially are 
used : 

a. Index construction 

1. In general, indices will be calculated by 
finding the arithmetic mean of the scores 
attained by a respondent on a number of 
items which are designed to measure a 
common characteristics. 

2. Clusters of highly intercorrelated indices 
within the same general category will be 
identified. The same general strategy 
used in index construction will be applied 
but in this case, the cluster scores will 
be based on the means of inter-related 
indices; in effect each such score will be 
an index of indices. 
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b. Correlational and multiple classification pro- 
cedures will be used to examine the relation- 
ship which exists between pairs of variables. 



c. Longitudinal analysis 

1. Comparison of measurements taken at two 

or three different points in time to assess 
causal directions underlying relationships. 

2. Assessment of proportion of total change 
taking place in each interval. This will 
be used in examination of traits which are 
believed to be developing in a systematic 
fashion. 

2. Descriptive statistics used to examine descriptive 

data include means, standard deviations and response 
distributions. 



Even though the study covers only 10th grade males in 
public schools, it produces associations that are con- 
sistent with other studies of a more general nature. 
This study was carefully designed and implemented and 
the analysis was done in such a way as to permit examin 
ation of principal association with and without adjust- 
ment for other major associations. The presentation of 
data in Volume II is particularly clear. 
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Relevance to NAEP Study 
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James S. Coleman at at. 

Equality of Educational Opp^vtunitn. 

National Center for Educational Statistics, Department 
of Health, Education, and Welfare, Washington, D.C., 
1966. 

Objectives : 

1. To discern possible relationships betv/een students' 
achievement and the kinds of schools they attend. 

2. ^<jro determine the extent to which the racial and 



3. To determine whether or not the schools offer equal 
educational opportunities in terms of a number of 
criteria which are regarded as good indicators of 
educational quality (e.g., number of laboratories, 
textbooks, libraries, etc.; characteristics of the 
teachers and characteristics of the student bodies) . 

4, To determine how much students learn as measured by 
their performance on standardized achievement tests. 




ethnic groups are segregated from one another in 
the public schools. 
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study Design: 



1. Purpos^e is descriptive, tc provide estimates for a 
number of school, teacher, and st';.rlent character- 
istics in public schools across the nation; to pro- 
vide separate estimates for urban and rural 
localities in major geographic regions; to provide 
reliable estimates so that comparison can be made 
between Negro and other minority students with 
white students. 

2. Data collected from a cross-sectional suixrey of 
public elementary and secondary schools in the 
United States. 

3. Survey data collected from: a) student question- 
naires and tests of verbal ability, reading, and 
mathematics; b) teacher, principal, and superin- 
tendent questionnaires . 

4. Target population: elementary and secondary school 
students in U.S. public schools? teachers, princi- 
pals, and superintendents in U.S. public schools. 

5. Experimental population: Stratified sample of 
public elementary and secondary schools in the 
United States. 

Sampling Procedures: 

1, Sampling design based on a two-stage probability 

sample of the public schools in the United States. 
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2. Primary sampling unit (PSU) used was 2,883 counties 
of which 209 were metropolitan areas and 2,674 
were counties located outside metropolitan areas* 

Counties were chosen because (1) census and other 
descriptive data were readily available for counties 
and (2) the county more than local school districts 
would provide greater internal heterogeneity which 
is more efficient. 

3. Counties were then assigned to one of two groups, 
metropolitan or nonmetropolitan, according to 
whether they were included in a standard metropoli- 
tan statistical area (SMSA) or not. 

4. The groups were then stratified by geographical 
location and by the percentage of non-whites in the 
PSU. The boundaries for the percentage nonwhite 
categories were set at: 

a. 70% and over 

b. 30 - 70% 

c. 10 - 30% 

d. Under 10% 

In nonmetropolitan counties the last category was 
broken down into: 

1. Estimated nonwhite enrollment of 100 or 
more 

2. Estimated nonwhite enrollment under 100 
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5. Within each county and metropolitan area that was 

selected in the first stage, a listing of all public 
secondary schools with the 12th grade was obtained 
from the inventory of school plants. These listings 
were sent to the various State departments of edu- 
cation where the percent nonwhite enrollment in 
each school was indicated . 

The secondary schoolLs were then stratified into five 
groups according to nonwhite enrollment: 



a. 


75 


.1 - 100% 


b. 


50 


.0 - 75% 


c. 


25 


.1 - 50% 


d. 


10 


.1 - 25% 


e. 


0 


- 10% 



6. Estimated 12th grade enrollment took into account 
differences in grade span. 

7. Feeder schools: For each secondary school selected 
in the sample, the lower grade schools which feed 
their students into that secondary school were 
identified in addition to the percent of the feeder 
school students ordinarily attending the high 
school . 

Each feeder school sending 90% or more of its stu- 
dents to a sampled high school was selected in 
addition to feeder schools with probability equal 
to the percent of students who go on to the sampled 
secondary school. 
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VIX. Description of Samples: 

A. Populations 

Institutions 
Public secondary school 

principals 
Public elementary school 

principals 

otudents— all students in grades 3, 

6, 9, 12 in sampled schools; for 

grade 1, only half the fraction of 

students sampled in the other grades. 300,000 

Representativeness of Sample 

High School Principal Non-response: 66 of the 352 
secondary schools that did not return a principal 
questionnaire were subsampled and surveyed? infor- 
mation from 66 schools combined with questionnaires 
of 818 principals who had originally responded, from 
which estimates of nation-wide average high school 
characteristics for whites and nonwhites were 
calculated. 

School Characteristics: the information derived 
from the follow-up examination of 66 schools indi- 
cates that the overall availability of school 
characteristics stated in this report was under- 
stated by about 1 percentage point on the average. 
The difference between the availability to whites 
and nonwhites is also understated by about 1 per- 
centage point, on the average. 



Number sampled 
1,170 
3,223 
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student Response Errors: Revised questionnaires for 
grades 3/ 6, 9, and 12 consisting on.ly of items that 
could be verified by school records were administered 
in two districts in Tennessee, one metropolitan, one 
nonmetropolitan. This study concluded that pupils 
responded with reasonable accuracy to factual items 
about themselves, their schooling, and their family 
and homes. 

VIII. Variables Studied: 

Background variables: 

1. General background 

2 . Student Behavior 

3. Academic variables — dropout, grade point average, 
education history, aspirations 

4. SES of family 

5. Father's occupation 

6. Parents' education 

7. Academic expectations of parents 

8. Structural integrity of home (presence of father) 

9. Other language spoken in home 

Intermediate variables 



1. 


Studenx-. body characteristics 


2. 


Education of parents of classmates 


3. 


Faciliti^^s 


4. 


Special services 


5. 


Special programs 


6. 


Tracking 


7. 


Racial interaction 


8. 


Curriculum 



ERIC 



52 



9. Extracurricular activities 

10. Faculty attitudes (race-related issues, school 
policy) 

11. Principal and superintendent attitudes on policies 
and issues. 

Outcome Variables 

1. Student ability and achievement 

2. Student behavior 



Instruments and Measurement 

1. Achievement tests to measure accomplishments of 
school , criterion of achievement 

A. Grade 1 

1. picture vocabulary test - verbal ability 

2. association and classification tests ' non- 
verbal ability 

B. Grade 3 

1. picture vocabulary test - verbal ability 

2. classification and analogies - nonverbal 
ability 

3. reading and math tests 

C. Grade 6 

1, classification and analogies - nonverbal 

2 . reading and math 

\ 



3. sentence completion and synonym tests - 
verbal 

Grade 9 

1. classification and analogies - nonverbal 

2. reading and math 

3. sentence completion and synonym tests - 
verbal 

4. general information 

Test battery of published school survey tests: 

ETS Sequential Tests of Educational Progress (STEP 
reading and math) 

Inter-Araerican Tests of General Ability - nonverbal 
ETS School and College Ability Test (SCAT) - verbal 
comprehension 

Questionnaires 
A. Student 

1. Grade 1 (teacher fills in) 

race, family size, education of parents, 
father's work, items in home, behavior, learning 
ability, etc. 

2 . Grade 3 

sex, race, size of family, satisfaction with 
school bocks in home, academic standing, ex- 
pectations of parents, etc. 



D. 



3. Grade 6 

size of family, age, sex, race, education of 
parents, items in home, academic expectations 
of parents, racial composition of class, race 
of teacher, expected occupation, etc. 

4. Grade 9 and 12 

type of community grew up in, family size, 
foreign language in home, occupation of father, 
education of parents, items in home, academic 
program, educationeil expectations of parents, 
race of classmates and teacher, college plans, 
race of friends, extracurricular activities, 
grade average, track in English, work, counselor, 
opinion items, etc. 

Principal 

1. Existence of kindergarten, instructional pro- 
grams, facilities, books in library, achieve- 
ment tests, free lunches, equipment, tenure 
system, attendance, transfers and dropouts, 
degjtruction, drugs, nonwhites, programs, post- 
graduate, representatives from colleges, 
reputation of school, personal (e.g., degree, 
years experience, college, field, racial 
composition of his college, location of 
college), tracking in school, policy for slov* 
learners and advanced, special classes, opin- 
ion items on educational policy, e.g., bussing. 



C . Teacher 

1, Sex, race, parents education, major, degree, 

academic level of college, teaching experience, 
assignment to present school, salary, academic 
ability of students, satisfaction with job, 
racial composition of class, opinion on school 
issues, learning-related problems (e.g., home 
environment), counseling, test of verbal 
ability. 

D. Superintendent 

1. Assignment of teachers, qualification of 
teachers, current school issues, personal 
information (age, sex, degree, race), appointed 
or elected, system statistics, expenditures, 
etc. 



Statistical Procedures 

A. Multiple regression analysis 

B. Correlation matrices for major regressions 

C. Ratio-estimace procedure 

D. Analysis of variance 



E. Single-variable correlations 



Relevance to NAEP Study 

The Equality of Educational Opportunity survey, usually 
identified as the Coleman Study, was conducted in re- 
sponse to Section 402 of the Civil Rights Act of 1964, 
It was a large scale survey to investigate the associa- 
tions between outcomes and school factors such as 
facilities, services, staff, curricula, racial mix, and 
so on. In the course of the investigation, a great 
quantity of student characteristics, family background 
and community variables were collected and used in the 
analysis to adjust the differential outcomes for such 
background factors. Thus, the study provides basic 
data which were analyzed extensively in the Coleman 
report itself as well as subsequently by many others, 
notably Mayeske, Smith and Armour. Their findings 
are particularly relevant to the NAEP project. 




L.C. Comber and John P. Keevcs 

SoieriQe Education in Nineteen Countries , Internacional 
Studies in Evaluation I. 

Stockholm, Sweden: Almquist and Wiksell, 1973. 
Objectives : 

1. To devise cross-national measures of achievement 
in Science, based on a systematic analysis of the 
curricula in participating countries. 

2. To apply these measures to probability samples 
of students from different countries in order to 
to devise acceptably accurate national profiles 
of achievement. 

3. To determine how these profiles relate to school, 
home, and national, circumstances. 

4. To assess the state of Science curricula cross- 
nationally and determine relationships between 
Science curriculum innovations and student 
achievement. 
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5. To examine relationships between different 
teaching methods and student achievement in 
Science; especially the efficacy of teaching 
methods based on actual investigations carried 
out by the students in a scientific manner. 



V. Design of Study 

1. Study was a cross-national survey: Australia, 
Belgium (French- and Flemish-language speaking) , 
Chile, England, Federal Republic of Germany, 
Finland, France, Hungary, India, Iran, Italy, 
Japan, Netherlands, New Zealand, Scotland, Sweden, 
Thailand, and the United States. It was hoped to 
isolate curricula differences between under- 
developed countries and industrialized countries. 

2 . Sampling 

a. Primary sampling unit was the school, 
selected by probability according to various 
stratifications . 

b. In each country, subsamples of students were 
randomly selected from sampled schools: 

Population I — 10 year olds 

Population II — 14 year olds 

Population IV — in last year of secondary 

school 

Population IVS — Students majoring in Science 

in school. These students 
were not reported on in the 
report. 
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c. In each school/ a sample of teachers of 
Science was selected to provide pertinent 
information , 

d. For school information / a consensus was taken, 
so no sub-sampling was done. 

Instruments were designed to measure: 

a. Student achievement in Science 

b. Student achievement in five scientific areas; 
Earth S^-ience, Biology, Chemistry, Physics, 
practical application (laboratory work and 
experiment design) , 

c. Student attitudes toward school and Science 
study 

d. Student motivation, interests, etc. 

e. Student characteristics 

f. Home/family characteristics 

g. Teacher characteristics 

h. Learning enviornment 

i. School characteristics 

j. National eco-cultural-educational 
characteristics 



4. To assess differences between countries, data 

was aggregated for all sampled ivdividuals within 
the country; to assess differences between schools, 
data was aggregated on school level; to assess 
differences between students, data was aggregated 
across all students in the total sample as well as 
considering the student as the unit of analysis. 

Description of the sample 

1. For all countries except India, Iran, and the 
U. S., two-stage sampling was used. 

a. Schools were stratified and randomly sampled 
inversely proportional to size, region, type 
of school, or sex. 

b. Within each school, a random sample of 
students was drawn. 

2. India, Iran, and the United States used a three- 
stage sample, first sampling administrative units 
or school districts. 

3. Students samples were drawn from the following 
target populations (excluding those students in 
special classes, mentally or physically 
handicapped) ; 

Population I — students IG years' of age; all 

students taught by one general 
class teacher; 

Population II — students 14 years' of age; 

point before c'ropouts occur; 



Population IV — students in last year of secon- 
dary school; should be most 
sophisticated in Science know- 
ledge and application 

4. A sample of teachers for Population I and Science 
teachers for Populations II and IV were selected. 

5. Each sampled school was surveyed for school 
information . 

6. Weights were developed to adjust for over- and 
under-sampling . 

VII, Instruments and Measures 

1. To measure student achievement in Science: 

a. Test measuring scientific knowledge and 
abilities, with two attributes — subject 
matter (Earth Science, Biology, Chemistry, 
Physics, practical ability) and four 
behaviorial objectives (Functional Under- 
standing, Understanding, Application, Higher 
Processes) . 

b. Incorporated into this test were pencil ana 
paper practical items, to mee.sure practical 
abilities in Science (optional with ec\ch 
country) . 

c. Understanding the Nature of Science measure, 
to assess the students' ability to understand 
the nature and methods of Science as distinct 
from its actual content 
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d. Achievement tests in the four subject areas, 
for Science majors-Population IVS. 



2. To measure general cognitive ability 
a. Word Knowledge test 

3. To measure student motivation, interests, attitudes 
toward Science and Science classes 

a. Interest-in-Science scale (Population I, II, 
IV) 

b. Description of Science Teaching scale 
(Population I) 

c. Attitude towards School-Science scale 
(Population II, IV) 

d. Attitudes towai.xs Science in the World 
scale (Population II, IV) 

e. Description of Science Teaching — Textbook/ 
Experimental (Population II, IV) 

f. Description of Science Teaching — Laboratory 
work: Structured/Unstructured (Population II, 
IV) 

g. Student Questionnaire 

4. To collect data on student background, character- 
istics, home/family characteristics 

a. Student Questionnaire 

5. To collect information on the school 

a. Teacher Questionnaire 

b. Teacher Science Questionnaire 



o 
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c. School Questionnaire 



d. School Behavior index, filled out Jay students 
on the degree of flexibility /rigidity of 
school discipline 



e. Opportunity to Learn (Science) measure, 
completed by sampled teacher u, assessing 
whether the content of test items was covered 
in the school Science curriculum. 



f. Holding Power of the school, an index of the 
retention power of the school. 



5. To collect information on the teacher 

a. Teacher Questionnaire 

b. Teacher Science Questionnaire 



c, Likert-type attitude scales (Only sample of 
100 teachers) 



6. To collect information on countries 

a. National Case Study Questionnaire, completed 
by each lEA National Center, 



VIII. Variables studied 

Between Country 



Indices of economic development: GNP per capita, 
% GNP obtained from non-agriculture, % work force 
in non-agriculture, % women in all occupations 



ERIC 



65 



Opportunity to learn 



Holding lower 
Growth Scores 
Degree of excellence 
Between School 

Population I 

School Handicap Score: Father's occv\pation, 
Father's education, Mother's education, use of 
dictionary in home, number of books in the home, 
family size 

age of students in the sample 
sex of students in the sample 
type of school 
teaching methods 

grade level of students in the sample 
size of class 

students have regular science lessons? 

students have a textbook for Science? 

students make observations and do experiments? 

students make up own problems and design 
experiments? 

kindred variables: like school, school 
motivation, hours T.V. watched per day, 
hours spent reading for pleasure 

Population II 

Block 1: Home and Student Background 
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School Handicap Score (a composite variable) 
Age of Student 

Sex of Student (Male 1; Female 2) 

Block 2 : School or Program 

Type of program or course 
Tp)e of school 

Block 3 : Learning Conditions in the School 

Percent male teachers on school staff 

Number of laboratory assistants 

Sex of Science teachers in sample (Male 1; 
E'emale 2) 

Opportunity to learn 

School behavior scale 

Homework in P':ience (a composite variable) 
Hours homework per week 
Total Science homework per week in hours 

Study of Science (a composite variable) 
Currently taking Science 
Total study of Science in years 
Total hours current study of Science 

Block 4 : Kindred Variables 

Science interests and attitudes (a composite 
variable) 

Science interests and activities scale 

Science in the world scale 

Importance of Mathematics 

Expected education and occupation (a com- 
posite variable) 

Expected education 

Expected occupation 

Science reading (a composite variable 
available only for certain countries) 
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Reading Science and technical books 
and magazines 

Reading Science fiction 

Reading Science articles in newspapers 

Preference for Science and nature 
programs on TV or radio 

Po^-aiation IV 

Block 1 : Home and Student Background 

School Handicap Score (a composite variable) 

Block 2 : School or Program 
Type of program 
Type of school 

Block 3: Learning Conditions in the School 
Total enrollment 

Percentage teachers male and Science 
(a composite variable) 

Percentage male teachers in school 

Percentage Science teachers in achool 

Sex of Science teachers in sample (Male 1; 
Female 2) 

Number of ancillary v^^aff (a composite 
variable) 

Number of laboratory assistants 

Total number of ancillary staff 

Teacher training (a composite variable) 
Teacher's post-secondary schooling 
Extent of teacher training in Physics 
Extent of teacher training in Biology 

Teaching methods: Use of drill materials 

Opportunity to learn items tested 

Homework per week in hours 
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Total Science homework per week in hours 
Study of Science (a composite variable) 
Currently taking Science 
Total study of Science in years 
Total hours current study of Science 

\ 

Block 4 : Kindred Variables 

Science interests and attitudes (a composite 
variable) 

Science interests and activities scale 

Science in the world scale 

Importance of Mathematics 

Expected education and occupation (a 
composite variable) 

Expected education 

Expected occupation 

Between - Student 

Population I: 

Block 1 : Home and Student Background 

Home circumstances (a composite variable) = 
school handicap 

Age of student 

Sex of student 

Block 2 : Type of School 

Block 3 : Learning Conditions in the School 
Total enrollment 
Coeducation at Population 1 
Pupil teacher ratio 
Percentage male teachers 
Sex of teacher 
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Opportunity to learn 
Grade 

Class size 

Hours homework per week 
Regular Science lessons 
Science textbook available 
Observations and experiments 
Design own experiments 

Block 4 : Kindred Variables 
Like school 
. chool motivation 
1 Parents help with homework 
Hours W watched per day 
Hours reading for pleasure 

Population II 

Block 1 : Home and Student Background 

Home ciri5umstances (a composite variable) 
school handicap 

Age of student 

Sex of student 

Block 2 : Type of School or Course 
Type of program 
Type of school 

Block 3 : Learning Conditions in the School 
(Selected variables only) 

Total enrollment 

Percentage Science teachers 

Sex of teacher 

Teacher subject association membership 
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Part of time Gmployed 
Opportunity to learn 
Grade 

Hours homework per week 
Total years' study of Science 
Currently taking Science 

Block 4 : Kindred Variables 
Like school 
School motivation 

Science interests and attitudes (a composite 
variable) 

Science interests and activities (3) 

Science in the world scale (2) 

Importance of Mathematics (1) 
Expected education 
Hours reading for pleasure 
Science reading (a composite variable) 

Reading Science and technical books (1) 

Reading Science fiction (1) 

Reading Science articles in newspapers (1) 

Viewing Science TV programs (1) 

Population IV 

Block 1 : Home and Student Background 

Horra circumstances (a composite variable) = 
school handicap 

Age of student 

Sex of student 

Block 2: Type of School or Program 

Type of school 

Type of program 
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Block 3 : Learning Conditions in the School 
Total enrollment 

Teacher's post-secondary schooling 
Subject association membership 
Teacher training in Biology 
Involved in Science curriculum reform 
Students plan investigations 
Teacher preparation in school hours 
Grade 

Total years study of Science 

Science study and homework (a composite 
variable) 

Total hours study of Science (1) 

Total hours Science homework (1) 



Block 4 : Kindred Variables 

Science interests and attitudes (a composite 
variable) 

Science interests and activities (3) 
Science in the world scale (1) 
Importance of Mathematics (2) 

Expected education 

Expected occupation 

Hours reading for pleasure 

Science reading (a composite variable) 

Reading Science and technical books (2) 
Reading Science fiction 

Reading Science articles in newspapers (2) 
Viewing Science TV programs (1) 

Outcome variables 

r 

Student achievement in Science, cognitive 

Student achievement in ScicncG, practical 
application 
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student achievement in Science, affective 



IX. Statistical Procedures 

1. Correlation analyses 

2. Multiple regression analysis 

3. Factor analyses 

X. Relevance to the NAEP Study 

This is an important publication in the International 
Evaluation Series. It is one of the few studies that shows an 
important sex difference (for 12th grade students) . 
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John C, Flanagan and William W. Cooley 



Project Talent One^Year Follow-Up Studies, 

Cooperative Research Project No. 2333. School of Educa- 
tion, University of Pittsburgh, Pittsburgh, Pennsylvania, 
1966. 

Objectives : 

1. To examine the relationships between high school 
student characteristics and the educational and 
occupational choices students make one year after 
they are graduated from high school . 

2. To examine the nature of career development of 
American youth — the experience of employment, the 
nature of job satisfaction, and the nature and 
extent of post-secondary education. 

3. To prepare a comprehensive counseling guide indi- 
cating the patterns of aptitude and ability that 
are predictive of success and satisfaction in 
var^.us post-secondary career/educational choices. 

4. To provide a better understanding of the educational 
experiences which prepare students for their 
lifework . 

5. To develop an inventory of human resources. 



6, To develop a set of standards for educational- 
psychological measurements. 

V. Design of the Study: I 

1. Project TALENT is a 20-year longitudinal survey of 
school students who were tested by TALENT in 1960 
while in grades 9, 10, 11, and 12. These Follow- 
up Studies examined various career/educational 
aspects and characteristics of each grade one year 
after the students had been graduated from high 
school . 

2. Data from 1960 were baseline data, which were used 
as the basic independent variables. 

3. Each Follow-up Study explored a different aspect 
of post-secondary career/educational c\lternatives : 
post-secondary education choice; differences among 
college students; post-secondary work choice; 
stability of career plans; predicting career 

plan changes; redefining career plan groupings. 

4. Sampling ; Using the TALENT 1960 survey population 
as its base, the Follow-up Project attempted to 
survey the entire population of each grade one 
year after it had been graduated from high school. 
A special effort was made to avoid response bias 
by subsampling among initial non-respondents to 
this follow-up survey and collecting data from 
them through various agency means. 
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5. 



Instruments were constructed to measure student 
ability, knowledge, interests, satisfactions, 
plans, home factors, and school activities. 



6. A trait and factor approach was used. Traits were 
measured in school, and relationships were sought 
between traits and subsequent vocational/educational 
behavior . 



Descriptioft of the Sample: 



1. a. The TALENT sample was of all students in grades 

9-12 attending, in 1960, between 4-5% of all 
secondary schools in the United States. 

b. Schools were selected by a stratified random 
sampling of all senior high schools and 
associated junior high schools, stratified by: 
category of school (public, parochial, private) 
geographical area; size of senior class (public 
schools only) ; and retention ratio (public 
schools only) . 

c. 1225 schools participated, add data was col- 
lected from approximately 400,000 students. 

2. a. The Follow-up Project attempted to survey all 

students in the respective grade one year after 
it had been graduated from high school. 
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b. The numbers o"f responses, including follow-up 
of initial non-respondents were: 



Grade 12 62,404 

Grade 11 48,404 

Grade 10 54,299 

Grade 9 4 7,4 70 



Instruments and Measures: 

1. Instruments consisted of a two-day battery of tests 
and questionnaires: TALENT Test Battery 

Student Information Blank Questionnaire 

2. Instruments were designed to measure: 

a. students' specialized aptitudes 

b. students' general ability 

c. students' interests and temperament- 

d. home background 

e. plans for the future 

3. These various measures were combined into two major 
trait categories: 

ability (general intelligence, aptitudes, 

knowledges) 
motive (needs, interests, life style) 
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4. The ability battery consisted of 60 distinct mea- 
sures; the motive battery consisted of 27 different 
measures. These measures were the pool of infor- 
mation from which pertinent measures were extracted 
to answer the varying questions posed in the Follow- 
up Project. 

5. An additional variable — SES — was made up by scaling 
information from 9 student background variables. 



VIII. Variables Studied: 

Background variables 
Student sex 

Family SES level: father's education, mother's 
education; father's occupation; number of books in 
the home; student has his own room; appliances; 
T.V,, radio, etc.; family income; value of home. 

Environmental Stability: number of school changes; time 
since last school change; number of school days absent; 
hours per day spent studying; time lived in the same 
community. 

Student ability 
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Aptitude t 



The IS tests in the TALENT battery which can be class- 
ified as aptitudes are: 

R-211 Memory for Sentences: the ability to memorize simple 

descriptive statements and recall a missing word 
when the rest of the sentence ij provided sometime 
later. (16 items) 

R-212 Memory for Words:' the ability to memorize foreign words 

corresponding to common English words. (24 items) 

R-220 Disguised Words: the ability to become used to "strange" 

modes of spelling ordinary words, i.e., the puzzling 
out from context and appearance the meaning of a 
word which is vaguely reminiscent of a familiar 
English word. (30 items) 

R-240 VJord Functions in Sc^ntences: a measure of sensitivity 

to grammatical structure which does not employ 
the terminology of grammar; the ability to under- 
stand the structure of a sentence and to recognize 
the function of each word or phrase in the sentence. 
(24 items) 

R-250 Reading Comprehension: the ability to comprehend written 

materials; the subject reads a passage and then 
answers questions about it, referring back to the 
passage as often as he likes, (48 items) 
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Creativity: the ability to find ingenious solutions to 
a variety of practical problems. (20 items) 

Mechanical Reasoning: the ability to visualize the 
effects of the operation of everyday physical 
forces (such as gravitation) and basic kinds of 
mechanisms (for instance, gears, and pulleys, 
wheels, springs, levers). (20 items) 

Visualization in Two Dimensions: the ability to visu- 
alize how a figure would look after manipulation 
in three-dimensional space, by folding a ^lat 
figure to make a three-dimensional figure. (16 items) 

Abstract Reasoning: the ability to determine a logical 
relationship or progression among the elements of 
a complex nonverbal pattern, and to apply this 
relationship to identify an element that belongs 
in a specified position in the pattern. (xo items) 

Arithmetic Computation: the ability to add, subtract, 
multiply, and divide whole numbers quickly and 
accurately. (72 items) 

Table Reading: the ability to obtain information from 
tables quickly and accurately. (72 items) 

Cj.erical Checking: the ability to compare pairs of names 
to determine quickly and accurately whether they 
are identical. (74 items) 
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Object Inspection: the ability to spot differences in 
small objects quickly and accurately when comparing 
them visually. (40 items) 

Preferences: the ability to make a rapid choice of one 
from each of many adjectival pairs indicating the 
kind of friend the subject would prefer to have. 
(166 items) 

Knowledge trait s; 

Spelling: the ability to spell fairly common words. (16 
items ) 

Ca? i. ualization r knowledge of the rules of capitaliza- 
tion and how to apply them. (33 items) 

Punctuation: knowledge of the appropriate use of all 
standard punctuation marks, with spf^cial emphasis 
on sentences. (27 items) 

English Usage: the ability to recognize which of sev- 
eral ways of expressing something is preferred 
usage. (25 items) 

Effective Expression: ability to recognize clear, con- 
cise, smooth prise expression of an idea. (12 
items) 

Arithmetic Reasoning: the ability to solve arithmetic 
problems, with no emphasis on computing skill. 
(16 items) 
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Introductory *:athematics : knowledge of elementary 
algebra, fractions, deicmals, per cents, square 
roots, intuitive geometry, and elementary measure- 
ment formulas; topics usually taught up to and 
including grade 9, (24 items) 

Advanced Mathematics: knowledge of plane geometry, 
solid geometry, algebra, trigonometry, analytic 
geometry, and introductory calculus; topics 
normally taught in gradf?s 10-12 in college 
preparatory courses, (14 items} 



Information traits: 



Screening: a test of extremely basic, simple knowledge 
designed to identify mentally retarded, functional 
illiterates and uncooperative students. (12 items) 



Vocabulary (21 items) 


R- 


107 


Literature (24 items) 


R- 


108 


Music (13 items) 


R- 


109 


Social Studies (24 items) 


R- 


■110 


Mathematics (23 items) 


R- 


■111 


Mechanics (19 items) 


R- 


■140 


Farming (12 items) 


R- 


■141 


Home Economics (21 items) 


R- 


■142 


Sports (14 items) 


R- 


■14 3 


Art (12 items) 


R- 


•144 


Law (9 items) 


R- 


■145 


Health (9 items) 


R- 


■14r. 


Engineering (6 items) 


R- 


•147 



Physical Sciences (18 items) 
Bir, logical Sciences (11 items) 
Scientific Attitude (10 items) 
Aeronautice and Space (10 items) 
Electricity and Electronics 
(20 items) 

Practical Knowledge (4 items) 
Clerical (3 items) 
Bible (15 items) 
Colors (3 items) 
Etiquette (2 items) 
Hunting (5 items) 
Fishing (5 items) 
Outdoor Activities (other) 
(9 items) 
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Architecture (6 items) R-148 Photography (3 items) 

Journalism (3 items) R-149 Games (sedentary) (5 items) 

Foreign Travel (5 items) R-150 Theater and Ballet (8 items) 

Military (7 items) R-151 Foods (4 items) 

Accounting (10 items) R-152 Miscellaneous (10 items) 



Needs : 



Sociability (12 items) 
Social Sensitivity (9 items) 
Impulsiveness (9 items) 
Vigor (7 items) 
Calmness (9 items) 
Tidiness (11 items) 
Culture (10 items) 
Leadership (5 items) 
Self-Confidence (12 items) 
Mature Personality (24 items) 



Interests : 



Physical Science, Engineering, Mathematics (16 items) 

Biological Science, Medicine (8 items) 

Public Service (11 items) 

Literary, Linguistic (16 items) 

Social Service (12 items) 

Artistics (7 items) 

Musical (5 items; 

Sports (8 items) 

Hunting, Fishing (3 items) 

Business Management (14 items) 

Sales (6 items) 

Computation (10 items) 
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Office Work (7 items) 
Mechanical , Technical (15 items) 
Skilled Trades (18 items) 
Farming (7 items) 
Labor (10 items) 



Life Style! * 



No scales set up as of date of report, 



Outcome variables 



1. Educational choice: 



Males 

four-year college 
junior collego 
Armed Forces school 
technical institute 
trade/apprentice 

school 
no post-secondary ed 



Females 

four-year college 
three-year nursing school 
junior college 
secretarial/business school 

* trade school 
no post-secondary ed 



2 • Colleges selected : 

Boston University 

Bradley University 

Carnegie Institute of Technology 

Columbia University 

Cornell University 

Drake University 

Drexel Institute of Technology 

Duke University 

Fairleigh Dickinson University 
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George Washington University 

University of Hartford 

Hofstra University 

Long Island University 

Massachusetts institute of Technology 

University of Miami, Florida 

New York University 

Northeastern University 

Northwestern University 

Pace University 

University of Pennsylvania 

University of Pittsburgh 

Polytechnic Institute of Brooklyn 

University of Rochester 

Stanford University 

Syracuse University 

Temple University 

Washington University (St. Louis) 

Western Reserve University 

Major Fi.elds of Study in College: 



Males 


1. 


Mathematics 


2. 


Physical Science 


3. 


Biological Science 


4. 


Social Studies 


5, 


English and Literature 


6. 


Languages and Fine Arts 


7. 


Psychology 


8. 


Education 


9. 


Engineering 


10. 


Business 


11. 


Agriculture and Forestry 
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Females 



1. 


Math and Physical Science 


2. 


Biological Science 


3. 


Social Studies 


4. 


English 


5. 


Languages and Fine Arts 


6. 


Psychology 


7. 


Education 


8. 


Business 


9. 


Home Economics 


10. 


Nursing 



4* Career Stability; (for those who did not go on to 
college) 

Business (managerial) 
General clerical 
Draftsman 
Salesman 

Electronic technician 

Electrician 

Machinist 

Carpenter 

Metal tradesman 

Paiiter 

Driver 

Printer 

Laborer 

Farmer 
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5. Stability of Career Plans 



1. 


Mathematician 


2. 


Physical scientist 


3. 


Biological scientist 


4. 


Engineer 


5. 


Physician 

/ 


6. 


Dentist ' 


7 . 


Nurse 


8. 


Pharmacist 


9. 


Psychologist, sociologist 


10. 


Social worker 


11 . 


Clergyman, etc. 


12 . 


Government 


13. 


Lawyer 


14 . 


Teacher 


15. 


Accountant 


16. 


Businessman 


17. 


Writer 


18. 


Artist, entertainer 


19. 


Engineering, scientific aide 


20. 


Aviation 


21 . 


Medical technician 


22 . 


Office worker 


23 . 


Sa. lesman 

w4 ^ W 0 AllUA 1 


24 . 


Armed Forcps 


25 . 


Protective 


26 . 


Skilled worker • 


27 . 


Structural worker 


28 . 


Housewife 


29. 


Barber, beautician 


30. 


Farmer 
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IX. 



statistical Procedures: 



5. 



6. 



1. 



3. 



Multivariate analyses of variance 

Multiple group discriminant analyses 

2 

Mahalanobis D analyses 
Classification probabilities analyses 
Scaled discriminant vector analyses 
Correlation analyses — multiple, canonical 



X. 



Relevance to the NAEP Study 



This is only one of a long series of publications on 
Project Talent. This one provides the principal 
analysis of the associations between outcomes 
(knowledge - abilities - interests, etc.) and back- 
ground factors. Most of the analyses were done on a 
sample of responses to the main project, but sample 
sizes still are large. This report also is one of the 
principal reports that analyses postsecondary choice in 
education and occupation. 
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I. Walter I. Garms 



II. An Approach to the Measurement of Educational Need: The 

Relationship Between Sooioeoonomio Char act eristics and 
Pupil Achievement i,% Basic Skills in Early Elementary 
Schoo I , 



III. Submitted to the New York State Commission on the Quality, 

Cost, and Financing of Elementary and Secondary Education, 
Albany, New York, August 1971. 



IV. Objectives: 

1. To examine the possible connections between socio- 
economic status of students and their performance 
on standardized achievement tests, in order to de- 
velop SES proxy measures for assessing cultural 
deprivation . 

2. To develop SES criteria measures that could be 
built into an accountability mechanism that would 
facilitate comparisons of effectiveness among schools. 

3. To develop SES criteria measures that could be used 
as an element in the governmental distribution of 
funds, on the basis Oi." educational need, to public 
schools . 

4. To test interactions among predictor SES variables 
in hopes of improving the predictive value of other 
than racial-ethnic factors. 
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V. 



Design of the Study: 



1. Study was an assessment and manipulation of test 
results of a sample of third grade students on the 
New York State Achievement Tests on Reading and 
Arithmetic, 1970. 

2. The present study was to test a larg*" sample of 
school-^ than did the author's previous study (Garms 
and Smith, "Development of a Measure of Educational 
Need and its Use in a State School Formula" (Albany: 
New York State Educational Conference Board, 1969)), 
for the purposes of improving the validity of this 
study and enhancing its political acceptability. 

3. Sampling: 

a. The school was the primary sampling unit, 
randomly selected from a universe of schools 
stratified by geographical location in New 
York State. 

b. Within each sampled school, twenty third-grade 
students were to be systematically subsampled. 
All pupil data were aggregated to school level, 

4. The dependent variable - low achievement on the 
New ii'ork State Achievement test in Reading plus 
low achievement on the Achievement in Arithmetic - 
was defined as the percentage of sampled students 
from each school who scored below the fourth stanine 
in reading plus the percentage of students who scored 
below the fourth stanine in arithmetic , 
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5. Test results were on record at the school. Other 
data collected in this study were: 

a. SES predictor items for each sampled student, 
aggregated to school level 

b. information obtained from the school principal 
on community and school attendance area 
characteristics. 

VI. Description of the Sample: 

1, A two-stage sample was used: 

a. Schools were stratified by geographic locale 

in New York State - New York City, New York tj:-iSA, 
Other Urban, Upstate SMSAs , Non-SMSA - and a 
random sample of 428 schools was selected. 

b. Within each sampled school, school officials 
were to select from some available source every 
nth student in the third grade, until they were 
able to obtain data for twenty students. These 
data on the twenty students were then averaged 
to obtain values for the school as a whole. 

c. The number of schools selected, th>^ number of 
usable returns, and the response rate, all by 
stratum, are: 



93 



stratum 



Sampled 



Usable Response 
Returns Rate 



New York City- 


90 


65 


72% 


New York SMSA 


87 


52 


69 


Other Urban 


75 


53 


61 


Upstate SMSA 


86 


63 


73 


Non-SMSA 


90 


68 


76 


Totals 


428 


301 


70.3' 



VIII. Instruments and Measures: 

1. Student achievement was measured in terms of third 
grade results on the 19 70 New York State Achievement 
tests in Reading and Arithmetic. 

• 

2. SES variables were obtained from the school records 
of sampled third grade students. Some items may have 
required the school to contact the pupil's parents 
for information. 

3. A questionnaire, completed by the principal, collected 
his perceptions of school area parental income level, 
character of housing of the school attendance area, 
and dominant characteristic of the community. The 
last was used as a possiDle stratification device, 

not as a variable. The other two variables were 
eliminated because the responses were to arbitrary, 
consisting of the principal's unsupported opinion. 
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Variables Studied: 



Intermediate variables and Independent variables 

a. Racial-ethnic category: Black, Puerto Rican, other 

b. Broken home: not living with both parents 

c. Eligible for free lunch program 

d. Level of parents' education 

Q. Overcrowded home: more people living in dwelling 
than there are rooms 

f. Dwelling owned or rented 

g. Mobility from school to school since entering 
elementary school 

h. Parents' occupation 

Outcome variable 

Third grade achievement level: for each school, per- 
centage of twenty students scoring below the fourth stanine 
in reading achievement plus the percentage of student 
scoring below the fourth stanine in arithmetic achievement, 
as measured on the 1970 New York State Achievement tests. 

Statistical Procedures: 

1. Correlation analyses 

2. Multiple regression analyses 

3. Step-wide regression analysers 

4. Linear regression analyses 

5. Double log regression analyses 

6. Chow test for homogeneity of regressions 



X. 



Relevance to the NAEP Study 



This is a large-scale statewide assessment where the 
background data were largely obtained from school records.' 
Unfortunately (for our purposes) the school rather than 
the pupil is the unit of analysis. The application of 
associations between background and achievement to the 
setting of accountability standards is an interesting one. 
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Xntemationa^. Study of Aahievmcnt in Mathematico i A 
Ccmpaviaon of Twelve Countries (2 Vols.). 



III. Stockholm, Sweden: Almquist and Wiksell and New Yoarks 

John Wiley & Sons, 1967, 



IV. Objectives: 

1. TO determine the influence of societal changes in 
patterns of living and the development of indus- 
trial and technical products upon mathematics 
teaching and learning. 

2. To locate differences in various types of school 
systems both between and within countries on per- 
formance in mathematics on the basis of both system 
input and output variables. 

3. To investigate the effect of school organization, 
selection procedures, and differentiation upon 
students' mathematics performance and attitudes 
toward mathematics. 

4. To compare the differential effects of curriculum 
and methods of instruction on students' perform- 
ance in and attitudes towards mathematics. 
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V. 



Design ot study: 

!• Cross-national survey 



2. Four-year period from June, 1961 to December, 196S. 

3* Actual test construction of instruments began in 
March, 1962 and the cross-national testing was 
accomplished between January and June of 1964. 
Instruments were administered one time only, 

VI. Description of the Sample: 

1* Population: 

a. Countries: 12 countries were included in the 
study, as follows: Austriraia, Belgium, 
England, Finland, France, Germany, Isreal, 
Japan, The Netherlands, Scotland, Sweden, and 
the United States, Wichin each country four 
types of samples werci obtained based upon both 
experience with and type of mathematics in- 
struction. These samples are described in 
detail in the se^Jtion below. 

b. Each of the foi^r types of samples described 
below was tested in every country included in 
this study. The broad categories of students 
were distinguished and labeled "Population 

1" and "Population 3". Population 1 includes 
all students studying in institutions not 
primarily concerned with the goals of college 
or university preparation. Population 3 in- 
cluded those schools "from which university or 
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equivalent inatitutiona of higher learning 
normally S'oaruit their students" . Bach of 
these two groups was subdivided as fullowsj 



Population la: All pupils who were 13.0-13.11 years 

of age at the date of testing 
lb: All pupils at the grade level where 

the majority of pupils of age 13.0- 

13.11 arc found 
3a: All pupils "studying mathematics 

as an integral part of their course" 
3b: All pupils "studying mathematics as 

a complementary part of their studies" 



2. Sampling procedure: a two-stage probabilitr 
sampling procedure was used. 

a. A random sample of schools in each population 
described for each participating country was 
drawn . 

b. Within each school selected, a random sample 
of students was selcscted. 

c. Countries eventually* participating in the 
study were those expressing an interest in 
such research by sending representatives to 
the UNESCO Institute of Education Assemblies. 
The basic sampling problem was that of securing 
a representative sample of age and grade level 
groups in each country. 
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3 



q£ sample: 



a. The ranga of schools sampled was from 8 schools 
in France to 395 schools in the United States. 

b. The range of the number of students was from 
50 students (one of the four samples in the 
Netherlands) to 6,544 students in the United 
States. 

Approximate numbers: 

150,000 students 
13,000 teachers 
5,300 headmasters 

4. Sex: approximately equal numbers of males and fe- 
males with the exception of population 3a which 
consisted of 75% mal&3. 

5. Racial composition; not reported, 

6. Religious composition: not reported. 



VII. Instruments and Measurements: 

All of the instruments used in the study were developed 
by the research team specifically for this study. 

1, The International Project for the Evaluation of 
Educational Achievement (lEA) Mathematics Test 
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designed to measure student aptitudes and abilities 
in the following content areas; 

a. Basic and advanced arithmetic 

b. Elementary and intermediate Algebra 

c. Euclidian and Analytic Geometry 

d. Sets, Trigonometric and circular functions 
e* Analysis I Calculus, Porbability, Logic and 

Affine Geometry 

The Student Opinion Booklet, designed to determine: 

a. Student's description of mathematics teaching, 
and learning 

b. Student's description of school and school 
learning 

c. Student's attitudes toward mathematics as a 
process? difficulties in learning mathematics; 
and the place of mathematics in society. 

Student Questionnaire (STQ) , designed to measure: . 

a. student personal characteristics 

b. Student interests and future educational and 
vocational plans 

Teacher Questionnaire (TCHQ) , designed to determine 

a. Teacher experience, training and views 

b. Teacher ratings of the relevance of the lEA 
Mathematics test for their students 
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5. School Quast'.Aon/?&i>rt! i^iUQ) to:e School Administrator 
or Headmaster ^ d^itKii^rK'id t.> obtain information 
regarding?. 

a. The t','pe Kchcjox 

i3. Practices of rJie r-chool 

G« School peraonnel 

d. School financial situation 

6* . The National Information Questionnaire (NATQ), 

responded to by an expert on the educational system 
of each country, designed to gather qualitative and 
quantitative data on the structure of the educa- 
tional systera of that country. 



VIII, Vairables Studies: 

1, Student Variables: 

a. Aptitude and ability in: 

1, Basic and advanced mathematics 

2. Algebra 

3 , Geometry 

4. Trigonometry, Calculus, Probability, Logic 
and Affine Geometry 

b. Student's perceptions of mathematics teaching 
and learning 

c. Student's descriptions of school 

d. Student's attitudes toward mathematics as a 
process and the place of mathematics in society 

e. Student's attitudes: difficulties in learning 
mathematics 
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t, student's interests and oducational and voca- 
tional aspi^afcAona 
g. Personal chajractoristics (e.g., age, sex) 

2. Knvironniental Variables - B'amily 

a. Socioeconomic levol measured by level of 
Father's occupation and parents' level of 
education 

b. Place of residence (e.g., urban, rural) 

3. Environmental Variables - School 

a. Type of school (e.g., selective, comprehensive, 
non-coeducational ) 

b. General practices of the school (e.g., using 
inquiry-centered techniques, age of initial 
enrollment) 

c. School finances (e.g., per-pupil expenditure, 
teacher salaries) 

d. Teachers' experience, training and attitudes 
toward their school and teaching 

4. Environmental Variables - Community (Country) 

Structure of the educational system of the country 
(e.g., comprehensive, selective) 

Statistical Procedures: 

1, Descriptive statistics including central tendency 
and variability indices 
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2. Frequancy distributions 



3. Items analyses including difficulty and discrimina- 
tion indices 

4. Univariate 7 ratio comparisons 
5« Correlation matrices 

6. Multiple regression analyses 

X. Relevance to NAEP Study 

This is the first of a series of national assessment 
studies and sets the pattern for subsequent analyses. 
Like all of the International Studies the analysis dis- 
tinguishes between the school and the student as units 
for analysis. Also, the number of variables for which 
data were collected is extensive, particularly with 
regard to home/school environment. 
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Wallace M. Cohen, and Carl E. Wisler 

4 Study of the Achievement of Our Nation*s Students, 
<See Section X for additional reference raaterials used 
in the analysis.) 



III. DHEW Publication m. (OE) 72-131. Office of Education, 
Department of Health, Education, and Welfare, Washington, 
D.C. , 1973. 

IV. Ob jec Lives: 

1. To re-assess and examine the data collected in the 
Coleman-USOE Study. This re-assessment is directed 
to the following general question: to what extent 

is individual student achievement in school associated 
with aspects of home background and school environment? 

2, To explore different aspects of family background 
and of the school as they relate to the achi'evemeat 
of students of different racial -ethnic and sex-group 
memberships and of students in different regions in 
the country. 

3.. To identify those aspects of the student's background, 
whether alone or in juxtaposition with school aspects, 
that play the largest role in student school achievement. 

4. To serve as a reference source by summarizing and 
displaying structural properties of the data and to 
show how these structural properties permit informa- 
tion to be obtained about the possible effects of 
family cackground and school influences on student 
achievement. 
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5» 1^0 develop a model of the eaueatlonal ptoe^gs as it 
jraiafees to student achievement. 



V. Design of the study: 

1. The study was a re -assessment and examination of the 
Coleman-USOE data and conclusions. 

2. No new samp3.ing was done but the obtained data was 
analyzed extensively. 

3. The unit of analysis was the individual student, 
using differences among students, differences among 
schools, and differences among students within a 
school . 

4. a. Items from the Coleman study were empirically 

scaled and empirically grouped into indices, 
so that the 400 items of the USOE study could 
be grouped into manageable units. 

b. Variables were placed into a conceptual set of 

two major divisions, each with two sub-divisions. 

1. Family Background 

l.a. Horn© Background-social structural 
aspects 

l.b. Family Process-attitudes of 
parents and student 
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2, SdhoDl Charaeteristlcs 



Student Body Gharadteristics - 
fitudeint attributes 
2.b. Comprohensiv© Set of School Variables 

c. Outcome variables we::e student achievement and 
student school attitudinal development. 

5, Instruments (in the Coleman study) were designed to 
measure: 

a. Student achievement 

b. Student attitudes 

c. Family environment 

d. Home environment 

e. School environment 

f. Teaching environment 

Description of tbf» sample: (See Coleman study) 

Instruments and Measures: (See Coleman study) 

Variables Studied: 
Background variables 

1. Student race/ethnic group men4>ership - white, Oriental, 
Puerto Rican, American Indian, Mexican -American, 
Negro, other. 

2. Scudent sex 
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3. Hdffion of gohool - Mafejfopolican, Non-matropolitan, 

4. Student grade in school - i, a, 6, 9, 3.2 
IntormodiatG var.lahlog 

1. Family Background 

a. Homo Background 

1. SES - parents' oducational level, father's 
occupation, place of residence, size of home, 
intellectual climate of home. 

2. Family structure and stahilicy - both parents 
at home, income of parents, mother works or 
not, residential mobility. 

b. Family Process 

1. Expectations for excellence - parents' expecta 
tions for student in school, student's own 
expectations . 

2. Attitude toward life - work ethic, opinion 
about efficacy of education in life, opinion 
about the rigors of life, estimation of the 
difficulty of learning. 

3. Educational plans and desires - parents' 
educational expectations for the student, 
student's own expectations, student's self- 
esteem with respect to school ranking. 
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school work, reading habits, T.V, -watching 
habits, truancy. 

Student Body Characteristics (for each grade level in school) 
SES 

Family structure and stability 
Racial/Ethnic group membership 
Expectations for excellence 
Attitude toward life 
Educational plans and desires 
Study habits 
Achievement 



Comprehensive Set of School Variables 

All but seven of the 31 variables are indices. There 
were no problems of measurement at the lower grade 
levels. 

Facilities 

Plant and physical facilities 
Instructional facilities 
Pupils per room 
Age of buildings 

Pupil Programs and Policies 

Tracking 
Testing 
Transfers 
Remedial Programs 
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Free milk and lunch programs 

Accreditation 

Age of texts 

Availability of tcixts 

Pupil -teacher ratio 

Enrollment 

School Personnel and Personnel Expenditures 

Principal's experience 
Principal's training 
Principal's college attended 
Principal's sex 

Principal's estimate of the school's reputation 

Specialized s'-aff and services 

Teacher's experience 

Teacher's training 

Teacher ' s r>ocio-economic background 

Teacher's localism 

Teacher's college attended 

Teaching conditions 

Teaching-related activities 

Preference for student-ability level 

Teacher's sex 

Teacher's Racial-ethnic group membership 
Teacher's vocabulary score 

Outcome variables 

1. Student achievement 

reading and arithmetic achievement 

reading comprehension and mathematics achievement 

general knowledge 
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2. Student school outcomes 

expectations of excellence 
attitude toward life 
educational plans and desires 
study habits 



IX, Statistical Procedures; 

1. Correlation analyses 

2. Square multiple correlations 

3. Multivariate commonality analyses 

4. Commonality analyses 

5. Regression analyses 

6. Sequential analyses 



X. Relevance to the NAEP Study: 

This document provides one of the principal sources of 
conclusions about individual student performance (in 
contrast to average school performance reported in 
A Study of Our Nation's Schools . Two auxiliary documents 
used in the analysis are: 

1. The appendices to the working paper entitled "A 
Study of Our Nation's Schools," (same authors). 
These appendices report simple and partial corre- 
lation coefficients, averages of criterion scores, 
etc. 
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2. Mayeske, Weinfeld, Beaton, Davis, Patters, and 

Hixon, "Stern response Analysis of the Educational 
Opportunities Survey Student Questionnaires," 
NCES Technical Note Number 64, April 1968. 

These reports are an important source because they 
provide the quantitative support for the conclusions 
reached in the published report. They also provide 
the basis for our computation of percent of variance ex- 
plained. 

The Mayeske, et al, reports as a group represent the 
most comprehensive source of the kinds of data we have 
looked for in this study. 
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George W. Mayeske, Carl E. Wisler, Ablert E. Beaton, Jr., 
Frederic D. Weinfeld, Wallace M. Cohen, Tetsuo Okada, 
John M. Proshek, and Kenneth A. Tabler 

A Study of Our Nation's Schools, (See Section X for 
additional reference materials used in the analysis.) 

Office of Education, Department of Health, Education, and 
Welfare, Washington, D.C., 1970. 

Objectives : 

1. To re-assess and examine the data collected in the 
Coleman-USOE study. This re-assessment is directed 
to the following general questions: what character- 
istics of the schools seem to be related to school 
outcomes and what aspects of the schools might be 
most important in producing these outcomes? 

2. To discover what differences among schools are 
related to school outcomes and how both are re- 
lated to the socioeconomic background and racial/ 
ethnic group membership of the students. 

3. To serve as reference source by summarizing and 
displaying structural properties of the data and 
to show the extent to which the structural proper- 
ties of the data will permit answers to be obtained 
about the possible influences that schools may have 
on their students . 



113 



To identify the percent of school outcome associated 
with aistinguishable influence of the schools' 
characteristics; the percent of school outcome 
associated with the distinguishable influence of 
the students' social background; and the percent 
of school outcome that could just as well be 
associated with either one. 



Design of the Study: 

1. The study was a re-assessment and examination of 
the Coleman-USOE study data and conclusions. 

2. No new sampling was done but the data obtained 
was manipulated many ways for the purposes of the 
present study. 

3. Data was aggregated to the school level, comparing 
differences between schools, in order to answer 
the general question: how do the schools' charac- 
teristics influence such things as the average 
achievement level of the students in school? 

4. a. Items from the Coleman study were grouped into 

indices of related items, so that the 400 items 
of the USOE study could be reduced to a manage- 
able number. The grouped indices were then 
divided into 3 major divisions: 

Students' Social Background 
Schools' Characteristics 
School Outcomes 
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b. Outcome variables were in terms of student 
achievement, student educational attainment, 
student attitudes. 

5. Instruments {in the Coleman study) were designed 
measure : 

a. student achievement 

b. student attitudes 

c. family environment 

d. home environment 

e. school environment 

f. teaching environment 

Description of the sample: (See Coleman study) 
Instruments and Measures: (See Coleman study) 

Variables Studied: 

Background variables 
Student sex 
Student age 

Student race/ethnic group membership 
community of residence 
Student grade 



Intermediate Variables 



STUDENTS' S0C:AL BACKGROUND 

Expectations for Excellence (Index 1) 

-mother's desire for child's academic excellence 
-father's desire for child's academic nxcellence 
-student's own desire to excel 
-teacher's expectations for student to excel 

Socio-Economic Status (Index 2) 

-type of community in which student has spent most 
of his life 

-nuiiODer of siblings / 

-number of rooms in the home 

-father's occupational level 

-father's educational level 

-mother's educational level 

-appliances in the home 

-reading materials in the home 

Attitude Toward Life (index 4) 

I 

-life condition 

-work for success 

-difficulty getting ahead 

-education in job 

-sacrifice to get ahead 

-want to change 

-learning problems 

-teaching rate 

-successful life 

-ability to do many things well 

-liked by classmates 
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Social Confidence (index 3} 



-outside work 
-social rating 
-success in life 
-tough job 

-ability to do many things well 

Family Structure and Stability (Index 5) 

-area in which student has spent most of his life 
-who acts as your father 
-who acts as your mother 
-family's source of income 
-mother's work 

-recency of change in school 
-frequency of changes in schools 

Educational Desires and Plans (Index 6) 

-father's desire for child's educational level 
-mother's desire for child's educational level 
-student's desire for higher education 
-student's plans for college 
-brightness 

-occupational level preferred 
-good student 

Study Habits (Index 7) 

-school discussions with parents 

-preschool reading 

-number of books read during summer 
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-number of hours watching TV 
-attitude toward school 
-study time 
•voluntary absences 

Classroom Behavior (Index 8) 

-gets along well with classmates 
-avoids disturbing classmates 
-arrives at school on time 
-shows desire to learn 
-shows good speaking vocabulary 
-pays attention in class 

-moves from activity to activity progressivf^ly 
-assumes responsibility 

SCHOOL CHARACTERISTICS 

Teacher 

Experience (Index 1) 
-age 

-number of years teaching 

-number of years teaching in this school 

-expects to remain in teaching until retirement 

Teaching Conditions (Index 2) 

-student effort 
-student ability 
-reenter teaching 
-prefer other school 
-school reputation 
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-school problems: 

External 
-school problems: 

Internal 
-ability grouping taught 

Localism of Background (Index 3) 

-area spcmg most of life 

-area graduated high school 

-area of undergraduate institution 

Socio-Economic Background (Index 4) 

-type/size of community spent most of life 
-father's occupational level 
-father's educational level 
-mother's educational level 

Training (Index 5) 

-highest degree held 

-certification 

-salary 

-tenure 

College Attended (Index 6) 

-undergraduate institution attended 
-highest degree offered by teacher's undergraduate 
institution 

-teacher's ranking of academic standing of under- 
graduate institution 
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Teaching-Related Activities (Indc;x 7) 

-attends summer institutes for teaching the cultur- 
ally disadvantaged 

-member of teachers associations 

-reads educational journals 

-hours a day spent in classroom preparation 

-hours a day spent in counseling (in addition to 
his official assignment) 

Preference for Student-Ability (Index 8) 

-type of high school pref<=rred 

-socio-economic background of students preferred 
-preference for high-ability students 

PRINCIPAL/SCHOOL CHARACTERISTICS 

Plant and Facilities (Index 1) 

-area of plant 

-central library 

-auditorium 

-gymnasium 

-cafeteria 

-athletic field 

-kitchen 

-infirmary or health room 

Principal's Experience (Index 2) 

-number of years as a principal 

-number of years as a principal in this school 

-years of age 
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principal's Training (Index 3) 



-highest degree held 
-salary 

College Attended (Index 4) 

-ranking of undergraduate institution 

-highest degree offered by undergraduate institution 

-location of undergraduate institution 

Instructional Facilities (Index 5) 

-number of volumes in the library 
-chop 

-biology labs 

-chemistry labs 

-physics labs 

-foreign language labs 

-typing rooms 

-movie projector 

-extracurricular activities 

Specialized Staff and Services (Index 6) 

-free kindergarten 

-art teacher 

-music teacher 

-speech teacher 

-mental health provisions 

-remedial reading teacher 

-number of guidance counselors 

-librarian 

-nurse 

-attendance officer 
-special classes 



Tracting and Ability Grouping (Index 7) 

-ability grouping or tracking 
-proportion of stu<^ents in highest track 
-proportion of students in lowest track 
-proportion of students moved to higher track 
-proportion of students moved to lower track 
-accelerated curriculum 

Frequency of Testing (Index 8) 

-frequency of intelligence testing 
-frequency of achievement testing 
-frequency of .''nterest testing 

Pupil Transfers (Index 9) 

-percent of pupil transfers in 
-percent of pupil transfers out 

Remedial Programs (Index 10) 

-percent of students in remedial math 
-percent of students in remedial reading 

Free Milk and Lunch Programs (Index 11) 

-percent of students who get free lunch 
-percent of students who get free milk 

Accreditation (Index 12) 

-state accreditation 
-regional accreditation 



Age of Texts (Index 13) 



-age of texts 

-date of reading books (elementary) or date of 
biology test (Secondary) 

Availability of Texts (Index 14) 

-tests provided 
-sufficient texts available 

Outcome Variables 

Student Achievement 

-verbal ability (all grades) 
-nonverbal ability (all grades) 
-reading comprehension (grades 3-12) 
-mathematics achievement (grades 3-12) 
-general information (grades 9, 12) 

Student Attitudes 

-expectati.ons for excellence 
-attitude toward life 
-educational plans and desires 
-study habits 

Student Educational Attainment 

-percent 12-grade graduates going on to college 
-percent 12-grade nonwhite graduates '^oing on to 
college 



•percent 12-grade graduates going on to vocational 
training 

-percent 12-grade nonwhite graduates going on to 

vocational training 
-percent 10-grade boys who drop out of school before 

completion of 12th-grade 

IX Statistical Procedures: 



1. 


Proportion of variance 


2. 


Criterion scaling 


3. 


Factor analyses 


4. 


Regression analyses 


5. 


Partition of multiple correlations 


6. 


Principal components analyses 


7. 


Varimax rotations 


8. 


Squared multiple correlation analyses 


9. 


Stratified regression analyses 



X. Relevance to the NAEP Study 

This document provides one of the principal sources of 
conclusions about school effects. Two auxiliary docu- 
ments used in the analysis are: 

1. The appendices to the working paper entitles "A 
Study of Our Nation's Schools," (same authors). 
These appendices report simple and partial corre- 
lation coefficients, averages of criterion scores, 
etc . 
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Mayeske, Weinfeld, Beaton, Davis, Fatters, and 
Hixon, "Stern response Analysis of the Educational 
Opportunities Survey Student Questionnaires," 
NCES Technical Note Number 64, April 1968. 



provide the quantitative support for the conclusions 
reached in the published report. They also provide 
the basis for cur computation of percent of variance 
explained. 

The Mayeske, et al, reports as a group represent the 
most comprehensive source of the kinds of data we have 
looked for in this study. 



These reports are an important source because they 




I. 



Alan C. Purves 



II. Litevature Eduaation in Ten Countrieo , International 
Studies in Evaluation II. 

III. Stockholm, Sweden: Almquist and Wiksell, 1973. 

IV. Objectives: 

1. To examine the relationships between facets of the 
Achievement-in-Literature study. 

2. To examine and explicate the relationships between 
these facets of achievement and major characteristics 
of students, their backgrounds, their curricula, and 
their instruction, 

3. To assess the differences among nations in the ways 
literature study is approached and taught, delineating 
the relationships between stated aims and actual 
outcomes, 

4. To examine the influences of schools and teachers 
on student achievement in literature study. 
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study Design: 

1. iStudy was a cross-national survey: Belgium (Prench- 
and Flemish-language speaking), Chile, England, 
Finland, Iran, Italy, New Zealand, Sweden, and the 
United States. 

2. Planning for the study b^.gan .in 1964; testing of 
instruments and design began in 1966. Questionnaire 
administration was completed in 1971. Instruments 
were administered only once. 

3. Sampling: 

a. Primary sampling unit was the school, selected 
within country by probability proportional to 
its enrollment. All data were aggregated to 
school level. 

b. For each country, probability samples were 
drawn from the defined target populations: 

Population II - students aged 14; last 
point before significant drop-outs occur 

Population IV - students in tri'^" last grade 
before entrance into the university 

c. All literature teachers and all teachers of the 
mother tongue in the school to be surveyed, 

d. Each school to be surveyed for school character- 
istics; addressed to school principal. 
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4. Instruments were constructed to measure: 

a. student cognitive achievement in literature 

b. student response patterns to literary works 

c. student transference of reading experiences to 
everyday life 

d. student characteristics 

e. teacher characteristics 

f. program characteristics 

g. school characteristics 

Description of the sample: 

1, All countries except the United States and Iran used 
two-stage sampling, 

a. First stage was to stratify schools according 
to sex and age of students, size of school and 
urban -rural character and then to draw a random 
sample of schools; 

b. Second stage was to draw a subsample of students 
within these schools, by class. 

2. United States and Iran used three-stage sampling. 

a. First stage was the sampling of communities and 
administrative units; 

b. Second stage was sampling of schools; 

c. Third stage was subsampling of students. 
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3. 



Responses of school principals were a consensus 
among all literature teachers and teachers of the 
mother tongue in the school surveyed. 



4. Numbers responding to survey: 



Number of schools 
responding 

Number of teachers 
responding 

Number of students 
responding 



Population 
II 



Total 



US 



1,092 145 
3,133 343 
23,392 3,344 



Population 
IV 



Total 



US 



884 115 
3,640 383 
29,014 2,472 



Instruments and Measures: 

1. Reading tests to measure students' work knowledge 
and reading comprehension. 

2. Student Attitude Questionnaire, to measure student 
interests in literature and transference of their 
reading experiences to their everyday lives . 

3. Student Questionnaire, to collect data on family 
characteristics and home environment. 

4. Student Questionnaire - literature, to measure 
students' interests and activities in reading. 

5. Teacher Questionnaire, to collect data on teacher 
characteristics, teaching practices. 

6. School Questionnaire, to collect data on school 
characteristics , 
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Variables studied: 



Background variables 
Student age 
Student sex 

Intermediate variables 

Student characteristics 

Home handicap scores Father's occupation, 
father's education; mother's education, use of 
dictionary in the home; number of books in the 
home; family size 

Study of literature measures 

Size of class and amount of instructioa 
Relation of literature to other mother-tongue 
instruction 

Emphasis given to the various genres 
Transference 

Age 
Sex 
Grade 

Preferred subjects in school 
Amount of homework per week 
Expected occupation 
Amount of reading for pleasure 

Items from the Student Questionnaire for the 
Reading Study: 

Reading preferences 

Genre preferences 

Literary medium preferences 
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Teacher characteristics 
Sex 
Age 

Degree of specialization 

Whether teaching subject of specialization 

Amount and type of education 

Amount of interest in professional matters 

Preferred mode of evaluation of student 

achievement 

Perception of determinants of curriculum 

School characteristics 
Locale 
Size 

Urban/rural 

Availability of cultural resources 
Type of program 
Sex make-up 
Curricula offerings 

Budgeting and per-student expenditures 

Outcome variables: 

Student verbal ability 
Student word knowledge 

Student ability to read passages for comprehension 
Student transference of reading to everyday life 

Statistical Procedures: 

1. Correlation analyses 

2, Multiple regression analyses 
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Relevance to the NAEP Study 

This is an important report in the International Evaluation 
Series. For our purposes, expression of the associations 
in terms of explained variance was particularly useful. 
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I. 



Marshall S. Smith 



II. "Equality of Educational Opportunity: The Basic 

Findings Reconsidered" 



III. Chapter 6 in Frederick Mosteller and Daniel P. Moynihan, 

eds., On Equality of Educational Opportunity. New York: 
Vantage Books, 1972. 



IV. Objectives: 

1. To re-examine the Coleman-USOE data and methodology 
that engendered the following five controversial 
conclusions : 

a. the relation of family background to achieve- 
ment does not diminish over the years of 
school 

b. family background accounts for a substantial 
amount of the school-to-school variation in 
achievement and, therefore, variations in 
school facilities, curriculum, and staff can 
only have a small independent effect. 

c. there is a small amount of variance explicitly 
accounted for by variations in facilities and 
curriculum. 
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d, although no school factor accounts for much 
variation in achievement, teacher character- 
istics account for more than any other. 

e. the social composition of the student body is 
more highly related to achievement, indepen- 
dently of the student's social bcxkground, than 
is any school factor. 

To focus on the validity of these five conclusions 
and on the effects that mistakes in the original 
analysis had on the Coleman Report. 

To examine the validity of two further inferences 
made by Coleman; that effects of school achievement 
increases over a student's years in school; that 
variations in school resources have a greater 
effect on the achievement of minority groups than 
on the achievement of whites. 

To measure the relationships between the schoolwide 
resources and student achievement. 

To eis^tend the original Report by exploring various 
interpretations of the data. 

To examine the policy implications suggested by 
this research. 
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V. 



Design of the Study: 



1. This study was a re-assessment and manipulation of 
the Coleman data. No new sampling was done and 

»»o new variables were introduced, 

2. The population was restricted, however, to only 
black and white students in grades 6, 9, and 12 in 
Northern schools only . 

3. Coleman variables were rearranged or omitted in 
some cases. 

4. Student verbal achievement was the outcome variable, 
with three schoolwide factors as independent 
variables: Facilities and Curriculum, Student 
Body, Teacher Characteristics. The Home Background 
factor was the control in assessing the relation- 
ships between student achievement and schoolwide 
resources . 

VI. Description of the Sample; 

1. The study made no changes in the Coleman sampling 
scheme . 

VII. Instruments and Measures: 

1. The study introduced no new variables into the 
study. 
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Variables studied: 



Background variables 
student grade 
studeiit race— white/black 

Intermediate variables 

Home Background Factor 

-urbanism of pupil and parents 

-parents' education 

-structural integrity of the home 

-home items (appliances) 

-number of reading items in the home 

-number of siblings 

-parents' interest in child's school experiences 
-parents' desires and expectations of child's 
success in school 

Student Body Factor 

-proportion of students who own encyclopedie 

-student transfers 

-attendance 

-average hours spent on homework 

-proportion who plan college 

-student body quality (rated by teacher) 

School Facilities and Curriculum Fcictor 

a. descriptive school measures: size, location 

b, physical resource measure: quanity of re- 
sources, per pupil expenditure systemwide, 
library volumes per student, number of science 
labs, guidance counselors 
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c. descriptive curriculum and program resources: 
comprehensiveness of cirriculum offerings, 
accelerated curriculum, tracking, movement 
between tracks, proportion of slow learners, 
extracurricula activities. 

Teacher Characteristics 

a. Ascribed: proportion of teachers in school 
who are white, average educational level of 
teachers ' mothers 

b. Achieved: average teacher verbal score, 
average teacher educational level, average 
localism of teacher 

c. Professional commitment and Preferences: 
average years teaching experience, average 
teachers' preference for type of student 
body. 

Outcome variable 

Student verbal achievement 

Statistical Procedures: 

1. Regression analyses 

2. Correlation analyses 

3. Multivariate analyses of v. lance 

Relevance to NAEP Study 

This study is an important reanalysis of the Coleman 
data with emphasis on the student as the unit of 
analysis . 




I. Robert L. Thorndike 



II. 



Reading Comprehension in Fifteen CcuntHes. International 
Studies in Education. 



III. 



New York: John Wiley & Sons, 1973. 



IV. 



Objectives : 



1. To study reading performance and the factors that 
are related to that performance in a group of 
countries, 

2. To develop and assess predictors for, and so 
account for differences in, reading ability, on 
three levels: 

a. differences among individual students in each 
country, without regard to the particular 
schools. 

b. differences in average performance among 
schools within the same country, 

c. differences among countries in overall aver- 
age achievement. 
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3. To examine background and school factors, within 
and among countries, that may account for 
differences in reading comprehension, 

4 . To compare results in Reading Achievement with 
other cross-national studies in Science and 
Literature. 

Design of Study: 

1, Study was a cross-national survey: Belgium (French- 
and Plemich-language speaking, Chile, England, 
Firland, Hungary, India, Iran, Isreal , Itlay, 
Netherlands, New Zealand, Scotland, Sweden, and 

the United States. 

2. Sampling: 

a. Primary sampling unit was the school, selected 
by probability according to various 
stratifications 

b. In each country, subsamples of students were 
randomly selected from sample schools: 

Population I — students 10 years old 
Population II — students 14 years old 
Population IV — students in last year of secon- 
dary school 

c. In each school, sample of teacher was selected 
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d. For school information, a consensus for the 
school was attained. 



3. Instruments were to measure: 

student reading comprehension through tests 
focusing on cognitive content of reading 
passages 

student characteristics 
background factors and home/family 
characteristics 
school factors 
teaching factors 
indigenous national factors 

IV. Description of the sample: 

1. Two-stage sampling was used in each country 

a. Schools were stratified by region, type of 
school, size of school and a random sample 
was drawn. 

b. Within each school, a random sample of students 
was drawn, usually selecting all individuals 
whose birthdays fell upon randomly selected 
days of the month. This selection was done 

by the lEA National Center of each country, 
rather than by the schools, to avoid school 
bias in selecting or excluding students. 



a. 



b. 
c . 



\ 



d. 
e. 
f . 
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Probability samples were drawn from the following 
target populations: 

Population I — students 10 years of age 
Population II — students 14 years of aye 
Population IV — students in last year of secondary 
school 



3. Weights were developed to adjust for over- and 
under-sanpling 



4. Teachers were selected within each school on random 
basis 



5. Each sampled school was surveyed for school 
information , 



6. Information was collected for: 



Number of 
Number of 
Number of 



students 
teachers 
schools 



Pop I 

34 ,344 
Not given 
1,670 



Pop II 

39 ,307 
Not given 
1,752 



Pop IV 

29,474 
Not given 
1,209 



Instruments and Measures: 

1. To measure student reading ability: 

a, conventional-type reading comprehension test, 
presenting a passage and asking questions 
about the passage, which is available for 
reference and re-reading 



b. a test of reading speed, consisting of a short 
paragraph ending with a question, and followed 
with three words, one of which is to be under- 
lined as answering the question 

c. a test of word knowledge, format of word pairs 
to be jduged as synonyms or antinyms 

To collect information on student characteristics, 
home characteristics, family characteristics, 
student interests, attitudes, and aspirations 

— Student Questionnaire 

To collect information on school facilities, ser- 
vices, curriculiim, pedagogical methods, teacher 
characteristics . 

a. School Questionnaire 

b. Teacher Questionnaire 

To collect information about national cultures and 
educational systems 

a. National Case Study Questionnaire, to be com- 
pleted by the lEA National Center for each 
country 

Student background information pooled for all 
individuals in a given country; school variables 
are related to differences in average reading 
achievement of students 



Variables Studied: 



Background variables: 

Student age 
Student sex 
Home Handicap index 

-Father's occupation, Father's education, Mother's 
education, dictionary in the home, size of 
family 

SES index 

-father's occupation (for Population I) 
-father's occupation, mother's education, father's 
education (for Populations II and IV) 

Intennediate variables: 

Home/Family 

-reading resources 

— dictionary in the home, wether newspaper 

received in the home or read by the student 
— number of books in the home 

-parental interest and involvement in schooling 
— parents' expressed interest in school 
— encouragement to read 
— encouragement to visit museums 
— parents help with homework 
— parents correct speech of student 
— parents correct writing of student 

Student characteristics 

— occupational aspiration 

— educational aspiration 

— literary medium preferences 
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--genre preferences 

—reading preferences 

--liking of school measure 

—motivation 

— literary interest 

—science interest (from lEA Science study) 

Teacher characteristics 

-sex 

-age 

-years' teaching experience 
-professional association 
-education 

-teaching specialization 
-teaching methods 

School characteristics 
-size 

-type of school 
-resources 

— library 

— class library 

— number of librarians 

— number of reading specialists 

— ancillary personnel 

— referral services for poor readers 
-expenditures per-student 
-teacher/student ratio 
-teaching practices 

— individualized instruction 

— class groups according to ability 

National cultural variables 
-economic development 



o 
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Outcome variables 



-student reading achievement 

— student reading comprehension 
—student reading speed 
— student word knowledge 

IX, statistical Procedures: 

1 . Correlation analyses 

2. Multiple regression analyses 

3. System step-wide regression e .'.^alyses 

4. Difficulty and Discriminant analyses 

5. Factor analyses 

X. Relevance to the NAEP Study 

Scores on reading achievement were used as an input 
variable in the analysis of International Evaluation 
results in Science and Literature. In that sense, 
one might consider reading achievement to be a measure 
of ability. By using reading achievement, the percent 
of variance explained was materially increased. 
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I. 



Dale Tillery 



II. "Scopes School to College: Opportunities for Post- 
Secondary Education (Abstract) " 

m 

III. In Clare Rose and James W. Trent, An Analytical Review of 
Longitudinal and Related Studies As They Apply to the 
Educational .Process — Research in Retrospect : Implica- 
tions for the Future, Vol. 4. Center for the Study of 
Evaluation, University of California Graduate School of 
Education, Los Angeles, California, December 1972. 

/ 

IV. Objectives: 

1. Determine the relationships that student character- 
istics, in terms of personal characteristics, family 
background, and educational opportunities, have with 
different school experiences, aspirations, attitudes, 
and decisions about post-secondary education. 

2. Determine the process (how, when, and why) and the 
stages of student educational and career decision- 
making during high school years; determine the in- 
fluences of parents, schools, and peers upon the 
nature of those decisions. 

3. Analyze the differences among defined clusters of 
students; analyze change and constancy of these 
groups over time; study those students who deviate 
from the peer groups on important variables. 
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4. Assess the congruence between students' perceived 
strengths and their stated aspirations and eluci- 
date as to how the students' view their decisions in 
retrospect. 

Study Design: 

1. Study was longitudinal, over a six year period, with 
test-retest on the same sample. Major testing points 
at grades 9, 12, and near the end of the first year 
of college. 

2. Data collected from school records, follow-up ques- 
tionnaires, and selective interviews for intervening 
years (grades 10 and 11). Follow-up data was ob- 
tained in 1968 for the 1966 9th grade sample; in 
1970 for drop-outs and other sub-groups. 

3. Cross-sectional data was analyjied as w^ll as data 
from randomly selected sub-groups of students who 
attended college and students who did not attend 
college. 

4. Students in cross-sectional studies were stratified 
by sex and educational aspirations. Aspirations 
were: leave school, graduate from high school, 
attend junior college or soire special technical- 
vocational school, graduate from a four-year college, 
seek a post-graduate college degree. 
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Sampling procedure: 

Multi-state proportional random-sampling procedure. 

a. Counties in each of the states were statisti- 
cally grouped into similar clusters on the 
basis of: 

1. median family income 

2. proportion of white collar workers 

3. white and nonwhite racial composition 

4. mobility of the population 

5. rate of school attendance of school age 
children , 

6. school size I 

7. ratio of students who go to college to 
high school graduates 

b. Counties were then randonly selected from with- 
in each cluster of counties in each of the four 
states • 

c. School districts / then schools, were randomly 
selected from within the selected counties so 
that there would be samples of grade 9 and 
grade 12 students larqe cnouah to meet the 
requirements for the iniMaJ sanr le sizes. 

d. Eight non-ovcrlaT"^ina a pr j clusters of 
students based on statisticci l ] y aofincd 1 actors 
were established at the borrinninr? the 



longitudinal study. The eight clusters are 
defined in reference to three dimensions: 

1. High/low "school ability" 

2. High/low family socioeconomic status 

3. High/low educational opportunity 

Limitation of study due to sampling: 

a. The authors state that caution should be used 
in making generalizations about students as a 
result of examining the data since some students 
were away when the testing was done and some 
students chose not to participate. It is not 
known what effect this may have upon the 
representativeness of the state samples. 

b. Loss of a large metropolitan school district in 
Massachusetts which chose not to participate 
was not replaced by otner volunteer schools. 
The public school systems are underrepresented 
in both grades 9 and 12 with respect to some 

of the characteristics of large urban areas. 

With respect to the four-state composite, how- 
ever, the size of the sample and range of types 
of schools sampled should reflect the attitudes, 
abilities and interests. 



VI. 



Description of Sample: 
1. Size of sample: 



9th graders 



12th graders 



California 8,204 7,757 

Illinois 14,338 10,881 

Massachusetts 11,673 9,793 

No. Carolina 21,846 12,555 

Total 97,047 56,061 40,986 



Throug' the use of appropriate theoretical models, 
it was determined that 3,954 students should be 
available for the final data collection in each of 
the four states. The initial sample sizes then 
took into consideration persistence rates and drop- 
out rates of students through each pq/-iod of 
schooling, college-going rates in the four states, 
and a general persistence rate during the first 
year of college. 

Units of sampling: Individual schools 

244 public schools 
5 5 non-public schools 

2. Population: All 9th and 12th grade students enrolled 
in high school for the spring semester, 1966 in 
California, Illinois, Massachusetts and North 
Carolina . 

Four states chosen on the basis of the following 
criteria : 

a. They have different traditions regarding com- 
mitments to public and private higher education. 
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b. They reflect the traditions and educational 
beliefs of the major regions of the natirn, 
although they cannot be considered statisti- 
cally representative of such regions. 

c. They represent leadership in major aspects of 
higher education. 

d. They have recent master plans for higher 
education. 

3. Racial composition: not reported. 

4. Religious composition: not reported. 

VII. Instruments and Measurements 

A. Student questionnaires were designed to measure 

1. Proficiencies and talents 

2. Home characteristics: SES, style of living, 
composition of members of household, psycho- 
logical environment of home. 

3. Patterns of identification with peer group 
members . 

4. Influence of others (e.g., perceived charac- 
teristics of others, parents, school personnel, 
ideal persons) 

5. Decision-making process including information- 
seeking behavior regarding education and career. 
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B. Standardized tests to measure aptitude, achievement- 
motivation, intellectual orientation and values, 
vocational interests, 

C. Principal questionnaires designed to gather basic 
data about schools and personnel, 

VIII. Variables Studied 

Background variables 

1. Physical health and development 

2. Race 

3. SES measured by Father's occupation; composition of 
members of household; psychological and physical 
environment of home. 

Intermediate variables 

1. Academic aptitude and achievement 

2. Achievement motivation 

3. Educational and vocational aspiration 

4. Interests, talents; values and beliefs; intellectual 
orientation 

5. Information-seeking behavior regarding education and 
career 

6. High school and college activities and attitudes 
toward those school experiences 

7. Peer affiliation and peer culture 

8. School curriculum; teachers; counselors 

9. Resources and services of school 

10. Community characteristics (e.g., SFS of population, 
library facilities, commitment to education) 

11. State influences (Master plans, financial aid, 
types of institutions available, aovcrnance of 
institutions 
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Outcome variables 

1. Persistence 

2. Transfer 

3. Evaluation of educational decisions 

Statistical Procedures 

1. Analysis of Principal Components 

2. Cross-sectional Analysis of Cluster Differences 

a. Analysis of variance and covariance 

b. Wilcoxon test for matched samples with ranks 

c. Bartlett's test for homogeneity of variance 

3. Cross-sectional Multivariate Discriminant Analysis 

4. Longitudinal Analysis of Clusters 

a. Chi square test for significance of changes 

b. Markov process for measuring changes and 
constancy 

c. Friedman's test 

d. Pearson's rank correlation coefficient 

e. Multiple regression methods 

5. Longitudinal Multivariate Discriminant Analysis of 
Clusters 

6. Analysis of Membership of Student Clusters 

a. Markov Analysis 

b. Multivariate analysis of variance 

2 

c. Hotelling's T 
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X. 



Relevance to NAEP Study 



The study is of relevance because it shows association 
between background measures and post-high school educa- 
tion. The data collected in motivation, attitude, in- 
terest, values, high school activities and peer culture 
are particularly rich. Observed associations should be 
meaningful even though the study does not represent a 
probability sample of the United States. 
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